One-Stop Shopping for MBARI Upper Water Column Data : installing DRDS and connecting to BOG

This page last changed on May 17, 2010 by mccann.

These steps document the process on setting up the DRDS on a server and then connecting that server (as read-only) to the BOG database.

Download dods.war from here: http://opendap.org/pub/dods/DODS-Java-1.1/1.1.7/dods.war
Install apache tomcat (tested with 6.0.18)
Drop the dods.war in the $TOMCAT_HOME/webapps directory and start tomcat.
This will create a dods directory in the webapps directory.
Shutdown tomcat
Download the JTDS driver from: http://jtds.sourceforge.net/ and install the jar file in $TOMCAT_HOME/lib

Edit the $TOMCAT_HOME/webapps/dods/WEB-INF/web.xml file.

Configure the DRDS servlet to connect to the BOG Datbase. The web.xml should look something like:

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.2//EN"
    "http://java.sun.com/j2ee/dtds/web-app_2_2.dtd">

<web-app>


    <servlet>
        <servlet-name>
            dts
        </servlet-name>

        <servlet-class>
            dods.servers.test.dts
        </servlet-class>

        <init-param>
            <param-name>DebugOn</param-name>
            <param-value>showRequest showResponse </param-value>
        </init-param>

    </servlet>

    <servlet-mapping>
        <servlet-name>dts</servlet-name>
        <url-pattern>/dts</url-pattern>
    </servlet-mapping>
    <servlet-mapping>
        <servlet-name>dts</servlet-name>
        <url-pattern>/dts/*</url-pattern>
    </servlet-mapping>


    <!-- MBARI's BOG Database connection -->
    <servlet>
        <servlet-name>
            bog
        </servlet-name>

        <servlet-class>
            dods.servers.sql.drds
        </servlet-class>

        <init-param>
            <param-name>JDBCdriver</param-name>
            <param-value>net.sourceforge.jtds.jdbc.Driver</param-value>
        </init-param>

        <init-param>
            <param-name>JDBCconnectionURL</param-name>
            <param-value>jdbc:jtds:sqlserver://solstice.shore.mbari.org/BOG</param-value>
        </init-param>

        <init-param>
            <param-name>JDBCusername</param-name>
            <param-value>read-only-username</param-value>
        </init-param>

        <init-param>
            <param-name>JDBCpassword</param-name>
            <param-value>read-only-username-password</param-value>
        </init-param>

        <init-param>
            <param-name>JDBCMaxResponseLength</param-name>
            <param-value>100000</param-value>
        </init-param>

        <init-param>
            <param-name>UseDataSetName</param-name>
            <param-value></param-value>
        </init-param>


        <init-param>
            <param-name>DebugOn</param-name>
            <param-value>showRequest  JDBC</param-value>
        </init-param>

    </servlet>

    <servlet-mapping>
        <servlet-name>bog</servlet-name>
        <url-pattern>/bog</url-pattern>
    </servlet-mapping>
    <servlet-mapping>
        <servlet-name>bog</servlet-name>
        <url-pattern>/bog/*</url-pattern>
    </servlet-mapping>

</web-app>

Now you need to specify the DDS, DAS, and the HTML template for accessing the data in the BOG database.
1. Create a directory named 'bog' under the $TOMCAT_HOME/webapps/dods/datasets directory.
2. Create three directories under the 'bog' directory
  1. das
  2. dds
  3. info

In the 'das' directory create a DAS file that matches the name of the table you want to connect to. For this example, I connected to the BCTD table so I created a file called 'BCTD'.

Attributes {
    BCTD {
        CTD_BOTTLE_ID {
        }
        CRUISE {
        }
        EXPD {
        }
        SEQ {
        }
        DEPTH {
        }
        BOTTLE {
        }
        PROJECT {
        }
        CTRB_ID {
        }
        PLATFORM {
        }
        RJDAY {
        }
        DATE_TIME {
        }
        JTIME {
        }
        YEAR_DATE {
        }
        DEC_LAT {
        }
        DEC_LONG {
        }
        SITE_ID {
        }
        PRESSURE {
        }
        CHL_GFF {
        }
        PHAEO_GFF {
        }
        FOFA_GFF {
        }
        VF_GFF {
        }
        VE_GFF {
        }
        SENS_GFF {
        }
        FO_GFF {
        }
        FA_GFF {
        }
        FLUOR_RTO {
        }
        CHL_1u {
        }
        PHAEO_1u {
        }
        FOFA_1u {
        }
        VF_1u {
        }
        VE_1u {
        }
        FO_1u {
        }
        FA_1u {
        }
        SENS_1u {
        }
        CHL_5u {
        }
        PHAEO_5u {
        }
        FOFA_5u {
        }
        VF_5u {
        }
        VE_5u {
        }
        SENS_5u {
        }
        FO_5u {
        }
        FA_5u {
        }
        TMP {
        }
        SAL {
        }
        CONDUCT {
        }
        SIG_T {
        }
        PARCOS {
        }
        PAR4PI {
        }
        TRANSMISS {
        }
        CHLA {
        }
        FLUOR {
        }
        NH4 {
        }
        NO2 {
        }
        NO3 {
        }
        SIO4 {
        }
        PO4 {
        }
        TCO2 {
        }
        O2 {
        }
        OXY_ML {
        }
        OXY_PS {
        }
        IRRD_490 {
        }
        IRRD_520 {
        }
        IRRD_555 {
        }
        ALTIMETER {
        }
        rdep {
        }
        sal2 {
        }
        temp2 {
        }
        cond2 {
        }
        pot_tmp {
        }
        OxyT {
        }
        OxyC {
        }
        TransV {
        }
        TransBeam {
        }
        POT_TMP2 {
        }
        OxyV {
        }
        ISUS_NO3 {
        }
    }
}

In the dds directory, create a DDS file that is also named after the table you will connect to. For this example, the file name is BCTD again and looks like (unlike the DAS file, this file should have no extension, i.e. it should be named just 'BCTD'):

Dataset {
    Sequence {
        Int32 CTD_BOTTLE_ID;
        String CRUISE;
        Int32 EXPD;
        Int32 SEQ;
        Int32 DEPTH;
        Int32 BOTTLE;
        String PROJECT;
        String CTRB_ID;
        String PLATFORM;
        Int32 RJDAY;
        String DATE_TIME;
        Float64 JTIME;
        Int32 YEAR_DATE;
        Float64 DEC_LAT;
        Float64 DEC_LONG;
        String SITE_ID;
        Float64 PRESSURE;
        Float64 CHL_GFF;
        Float64 PHAEO_GFF;
        Float64 FOFA_GFF;
        Float64 VF_GFF;
        Float64 VE_GFF;
        Float64 SENS_GFF;
        Float64 FO_GFF;
        Float64 FA_GFF;
        Float64 FLUOR_RTO;
        Float64 CHL_1u;
        Float64 PHAEO_1u;
        Float64 FOFA_1u;
        Float64 VF_1u;
        Float64 VE_1u;
        Float64 FO_1u;
        Float64 FA_1u;
        Float64 SENS_1u;
        Float64 CHL_5u;
        Float64 PHAEO_5u;
        Float64 FOFA_5u;
        Float64 VF_5u;
        Float64 VE_5u;
        Float64 SENS_5u;
        Float64 FO_5u;
        Float64 FA_5u;
        Float64 TMP;
        Float64 SAL;
        Float64 CONDUCT;
        Float64 SIG_T;
        Float64 PARCOS;
        Float64 PAR4PI;
        Float64 TRANSMISS;
        Float64 CHLA;
        Float64 FLUOR;
        Float64 NH4;
        Float64 NO2;
        Float64 NO3;
        Float64 SIO4;
        Float64 PO4;
        Float64 TCO2;
        Float64 O2;
        Float64 OXY_ML;
        Float64 OXY_PS;
        Float64 IRRD_490;
        Float64 IRRD_520;
        Float64 IRRD_555;
        Float64 ALTIMETER;
        Float64 rdep;
        Float32 sal2;
        Float32 temp2;
        Float32 cond2;
        Float64 pot_tmp;
        Float64 OxyT;
        Float64 OxyC;
        Float64 TransV;
        Float64 TransBeam;
        Float64 POT_TMP2;
        Float64 OxyV;
        Float64 ISUS_NO3;
    } BCTD;
} test;

If you create a file named BCTD.html and put it in the info directory, it will provide some HTML for the user that will give them more information about the BOG dataset. For example:

<h2>BOG Dataset</h2>
<h3>
<p>
This is the MBARI BOG dataset
</p>
<p>
Please go to the following for more information:
</p>
<p>
<a href="http://www.mbari.org/bog/"     > MBARI's Biological Ocean Group </a>
</p>
</h3>

In the info directory create a file named dods.servers.sql.drds.html with the HTML show below. This HTML will show up in an information page that users will see when they browse to the DRDS service for your BOG dataset.

<h2>DODS Relational Database Server</h2>

<p>
  This data is being served by a DODS Relational Database Server (DRDS), which
  is a somewhat unusual instance of a DODS server. All of the data served by the
  DRDS is stored in a relational database management system (DBMS). Queries from
  DODS clients are translated into SQL queries and sent to the underlying DBMS.
  The reuturned data is read by the DRDS and returned to the requesting client.
</p>

<p>
  Because the DRDS is essentially a front end to a DBMS, the DDS's have a somewhat
  different meaning than in other DODS servers. Each DDS served by the a DRDS
  represents a table in the underlying DBMS. Since queries to the DBMS will return
  an unknown amount of data, each DDS basically must contain a representation
  of the tables contents defined as a DODS <b>sequence</b>. Because of this it
  rarely makes sense for a client to request the entire &quot;dataset&quot;, as
  this request will return the entire contents of the table. When requesting data
  from a DRDS, it is best to get the dataset information (using the .info extension
  on the DODS URL) and then build a constrained request that just returns data
  that is actually desired. This doesn't mean that the entire table cannot be
  requested and sent, it is simply a caution that each dataset/DDS/table may in
  fact be very large.
</p>

<h3>Server Functions:</h3>
<p>
  <b>unique():</b> This function gets used in the selection part of the DODS
  constraint expression. It requires no parameters. Calling this funcion will
  cause the return to contain only unique rows. This is useful for sifting
  through numerous identical fields, for example station data that contains
  instrument names. This is NOT very useful if you are trying to retrieve a
  ship track...
  <br>Example:
  <pre>
          http://nasty.dods.url/server/dataset.dods?var1,var2&unique()
  </pre>
</p>


<p>
  <b>Regular Expressions:</b> The typical DODS server supports the full range
  regular expression syntax. THIS DODS server does not. Limited string matching is
  available. The wild card characters '.' and '.*' may used. The '[...]' notation
  for matching a range of characters may be used, but only to match a single
  character to a range of possiblities.
  <br>Examples:
  <pre>
          http://nasty.dods.url/server/dataset.dods?var1,var2&var3~=".*south.*"
          http://nasty.dods.url/server/dataset.dods?var1,var2&var3~=".south."
  </pre>
</p>

<p>
  <h3>NOTE:</h3> String fields in the DBMS may contain white space (spaces). In order
  for a constraint to match you must include the spaces. For example, if you
  wish to retrieve all data where a field called <i>location</i> has a value
  of "Southern Ocean", in your DODS URL you will have to represent the space
  between "Southern" and "Ocean" with the symbol "<b>%20</b>". Like this:
  <pre>    location="Southern%20Ocean"</pre>
  However, if the DODS URL is formed with a space like this:
  <pre>    location="Southern Ocean"</pre>
  It will not be handled correctly by the DRDS.
</p>

<p>Have fun!</p>

Now start up Tomcat and browse to http://localhost:8080/dods/bog and you will see your table with some links that you can explore.

Pros

Nice way to make a RDB look like a DODS server.

Cons

Not active (most recent release is 2004)
Will not compile against Java 1.5(note says that although I did not verify)
Not sure if I can make the backing RDB into a Grid dataset instead of just a Sequence.

Installed on elvis using above instructions on 18 May 2009

Example query for bottles with depths > 2000:

http://dods.shore.mbari.org:8080/dods/bog/BCTD.ascii?&BCTD.DEPTH%3E2000

Attachments:

BCTD (application/octet-stream)

BCTD.das (application/octet-stream)

BCTD.html (text/html)